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Abstract. We show that the decidability of an amplification of Hilbert's Tenth 
Problem in three variables implies the existence of vmcomputably large integral 
points on certain algebraic curves. We obtain this as a corollary of a new positive 
complexity result: the Diophantinc prefixes 3V3 and 33V3 are generically decid- 
able. This means, taking the former prefix as an example, that we give a precise 
geometric classification of those polynomials /gZ[u, x, y] for which the question 

3veN such that VxeN 3yeN with f(v,x,y)=Q1 

may be undecidable, and we show that this set of polynomials is quite small in a 
rigourous sense. (The decidability of 3V3 was previously an open question.) The 
analogous result for the prefix 33V3 is even stronger. We thus obtain a connection 
between the decidability of certain Diophantinc problems, height bounds for points 
on curves, and the geometry of certain complex surfaces and 3-folds. 



1. Introduction 

We derive new complexity-theoretic limits on what can be discerned about the 
set of integral points of a variety of low dimension. In particular, we exhibit a new 
family of decidable Diophantine sentences related to the remaining open cases of 
Hilbert's Tenth Problem. As a corollary, we obtain a Diophantine problem whose 
decidability implies the following surprising assertion: for a general algebraic plane 
curve {(x,y) G C | f(x,y) = 0}, it is impossible to express the size of the largest 
positive integral point as a Turing computable function of the degree and coefficient 
sizes of /. 

Finding such bounds is literally one of the holy grails of number theory. Huge, 
but nevertheless computable, upper bounds have already been found for a number of 
important classes of curves, such as curves of genus one | ]BC70| | , Thue curves ||Bak68 



hyperelliptic curves ||Bak69|| , superelliptic curves ||Bri84|| , and certain rational curves 



|Pou93| | fl For example, it is known that for any polynomial equation of the form 

y 2 = a + bx + cx 2 + dx 3 
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where a, fe, c, dGZ and a + bx + cx 2 + dx 3 has three distinct complex roots, all integral 
solutions must satisfy 

\y\ < exp((10 6 #) 106 ), 

where H is any upper bound on \a\, \b\, |c|, \d\ [ |Bak75| | .p| 

However, finding such bounds, even ones monstrously larger than those already 
known, for general algebraic curves has been out of reach for decades. Furthermore, 
the analogous question for algebraic surfaces, even in C 3 , has so far been addressed 
only through deep conjectures of Lang and Vojta [|Lan83|, |Voj87| . 

Our first main theorem relates the decidability of certain Diophantine sentences in 
four variables with the computability of upper bounds on the size of integral points 
on algebraic curves. So let us briefly recall Hilbert's Tenth Problem in n variables: 
"Decide whether an arbitrary /£Z[xi, . . . , x n ] has an integral root or not." 

We will denote this well-known Diophantine problem by 7t / TT > i{n). Similarly, the 
analogous problem where we wish to determine the existence of an integral root, 
with all coordinates positive, will be denoted by 7iT'Pn(n). We will also need the 
following closely related functions. 

Definition 1. For any subset i?CC closed under addition and multiplication, define 
the functions 

Big fin , Exact Cardan : Z[xi, ... ,x n ] — ► N U {0, oo} 

as follows: Let Sf be the hypersurface {(x\, . . . ,x n ) G C n | f(x\,... ,x n ) = 0}. 
Then Big fl (/) is the supremum of max{|n|, . . . , |r n |} as (ri, . . . ,r n ) ranges over 
{(0, . . . ,0)} and the set of all points of Sf in R n . Finally, ExactCard# jn (/) is the 
number of points of Sf in R n . 

It is not hard to see that the decidability of HTV^(n) implies the decidability 
of HTVz(n), so HTVn(n) is at least as hard as TiTVz{n). Similarly, the com- 
putability of Big N n (resp. ExactCardj^n) easily implies the computability of Big z n 
(resp. ExactCard^.n). Also, via brute-force enumeration, it is easy to see that 
ExactCard N n (resp. ExactCard Z n ) is computable iff Big N n (resp. Big z n ) is com- 
putable. However, we also have the following more subtle fact. 

Main Theorem 1. At least one of the following two statements is false; 

1. The function Big N2 is Turing computable. 

2. The Diophantine sentence 

3?iGN VxGN 3yGN with f(u,v,x,y) = 

is decidable in the special case where the underlying 3-fold Sf contains a surface 
which is a bundle of curves (each with a genus zero component) fibered over a 
curve C in the (u,v) -plane, where C has infinitely many positive integral points. 

2 For any integral point (x,y) S Z 2 , the quantity max{|x|, \y\} is usually called the height of 
(x,y). Also, the preceding height bound has since been considerably improved, e.g., [3ch92|. 
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In particular, HTVfq(3) is a special case of the problem mentioned in statement (2). 

The geometric notions mentioned above are clarified in section 2. Alternative classes 
of C for which Main Theorem 1 remains true are mentioned in sections 1.1 and 5. 
We note that Alan Baker has conjectured [ |J on8 1 , Section 5] that 7iTPz(2) is 



decidable. Thus the truth of statement (1) above would imply an algorithm for 
deciding TLTVn{2), and thus a positive answer to Baker's conjecture as well. We 
also point out that the computability of ExactCard^ (and ExactCardp^) is still 
an open question, in spite of the fact that explicit (albeit huge) upper bounds on 
ExactCard^ 2 are known in many cases | )Bom9U| , |Pou9ci| |. 



On the other hand, Z. W. Sun has proved that TiTV^H) is undecidable |Sun92a 
Also, Y. V. Matiyasevich has shown (the proof appearing in a paper of J. P. Jones 
Jon82|| ) that HTV^(9) is undecidable. However, although the decidability of HTVn(l 



and 7iTVz{l) is a simple algebraic exercise, the remaining cases of HTV^(n) and 
HTVz(n), as of mid- 1998, are still completely open. 

Our result above thus tells us something new about the next harder (and open) 
cases of TiTV^{n). 

Remark 1. The computability o/Big Zn (resp. Exact Card^ ny ) does not trivially 
imply the computability o/Big Nn (resp. Exact Card Nny ): It is possible for Big Zn (/) 
(resp. Exact Car dz <n ) to be infinite and thus give us no decisive information about 
the value o/Big Nn (/) (resp. Exact Car d^ >n ). 

This connection between height bounds and Hilbert's Tenth Problem points to an 
unusual possibility: The search for general effective height bounds for integral points 
on algebraic curves may be futile. Indeed, it would have perhaps been more interesting 
to prove the statement u HTVm(3) is decidable =>- Big N 2 is uncomputable," or better 
still, u H.TVz(3) is decidable =>■ Big Z2 is uncomputable." However, Main Theorem 
1 is at least a first step in this direction. We will comment further on strengthening 
Main Theorem 1 in the conclusion of this paper. 

While our first main result is negative in the sense that it implies undecidability 
for certain Diophantine sentences, its proof follows easily from our derivation of two 
positive results on Diophantine sentences. To describe these results, let us introduce 
the following notation: We say that "the Diophantine prefix 3v\/x3y is decidable" 
iff there is a Turing machine algorithm which decides the sentence 

3vVx3y with f(v,x,y)=0 

for arbitrary input / e Z[v,x,y], and where the quantification is over the positive 
integers. This notation extends in an obvious way to other combinations of quantifiers 
and variables such as 3v3y, 3u3v 3y, etc. Finally, by generic decidability, we will 
mean that a prefix is decidable when the input is restricted to an a priori fixed "large" 
set. This is made more precise below and in section 2. 
We will prove the following result. 
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Main Theorem 2. The prefix 3vVx3y is generically decidable. More precisely, it 
is decidable on the collection of those f for which the underlying complex surface Sf 
does not have an irreducible component which is a bundle of curves (each with a 
genus zero component) fibered over the v-axis. 

By simply considering those polynomials in Z[u, y], note that the prefix 3v3y (or, 
equivalently, HTVn(2)) is a special case of the prefix 3v\/x3y. It is also easy to 
see (cf. section 2) that the set of "hard" / omitted by our result above happens 
to include Z[u,y]. Furthermore, via theorem |] at the end of this section, we can 
algorithmically determine whether / satisfies the above hypothesis. 

It should of course be pointed out that the decidability of 3 V3 was a completely 
open problem. In fact, J. P. Jones ||Jon81|| has conjectured that the prefixes 3V3 



and 33 are equivalent. Put another way, this is the conjecture that 3V3 is decidable 
-<=>- HTVn(2) is decidable. So while we still haven't resolved the decidability of 3V3, 
we now at least know a geometric characterization of where any potential obstruction 
to decidability may lie. In particular, it follows from a fundamental result of algebraic 
geometry that our hypothesis rules out certain ruled surfaces, i.e., surfaces which are 
traced out by an infinite family of lines. The latter statement is also clarified in 
section 2. 

Our final main theorem is a seemingly paradoxical extension of the preceding result. 

Main Theorem 3. The prefix 3u3vVx3y is generically decidable. More precisely, 
it is decidable on the collection of f for which the underlying 3-fold Sf contains no 
surface which is a bundle of curves (each with a genus zero component) fibered over 
a curve in the (u,v) -plane. 

We can algorithmically determine whether / satisfies the preceding hypothesis as 
well, via theorem [| at the end of this section. 

The "near paradox" arises from the following result of Y. V. Matiyasevich and Julia 
Robinson.0 

The MR Theorem. [ MR73 ] The quantifier prefix 33V3 is undecidable, i.e., there 



is no Turing machine which decides for an arbitrary input f £ 1i[u, v, x, y] whether 
there is a (u, v) £ N 2 such that Vx 3y with f(u, v,x,y) = 0. ■ 

Thus, our generic decidability result is stronger for the prefix 33V3: We obtain a 
necessary geometric condition classifying those / for which the above Diophantine sen- 
tence is undecidable. It is easy to see (cf. section 2) that this set of "hard" / includes 
the prefix 3u3v3y (i.e., the problem TCTV^(3i)). However, this does not necessarily 
imply that the prefix 3u3v3y is undecidable — the undecidability of 3u3v~tfx3y may 
be due to other polynomials in our exceptional locus. We also emphasize that the 
set of exceptional / in Main Theorem 3 is strictly larger than the set of / considered 
in Main Theorem 1. 



3-- 



The paper [Jon81] contains many important results related to the MR Theorem, and for non- 



Russian readers may be a better reference than the original reference [MR74] 
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The proofs of Main Theorems 2 and 3 are not difficult conceptually, but rely upon 
results of C. Runge |[Run87| , |Aya91|| , C. L. Siegel |fSie29fl , and A. Schinzel |[Sch82 



on the distribution of integral points on curves. The necessary results are stated in 
section 1.1. The application of these results then relies on combining a geometric 
construction with a new, more effective characterization of genus zero for algebraic 
curves. The following definition makes this more precise. 

Definition 2. Suppose g G C[a 1; . . . , a m , x\, . . . , x n ] and let a:— (a 1; . . . ,a m ). If we 
choose constants in C for all the a^, we denote the corresponding specialization of g 
to a polynomial in C[xi, . . . ,x n ] by g a . For any g G C[aj, . . . , a m , xi, X2], we then 
define the genus zero locus of g, Q g , to be the set of all aeC™ such that S 9a has 
an irreducible component with geometric genus zero^ 

By a celebrated theorem of Siegel (cf. section 1.1), detecting genus zero for a given 
curve (in many cases) is equivalent to detecting the existence of infinitely many in- 
tegral points. So the following theorem, proved in section 3, may be of independent 
interest . 

Theorem 4. For any g G Q[ai, . . . , a m ,x, y], the locus Q g is a quasi-affine variety, 
and the equations (and inequations) defining Q g can be constructed effectively, e.g., 
by a Turing machine. 

For example, a special case which is easy to derive from the basic theory of elliptic 
curves |[Sil95|| is the following: If g:=a\y 2 + 02 + a^x + a^x 3 , then the zero set of g in 
C 2 has an irreducible component of genus zero iff 

a\ai(Aa\ + 2703^4) =0. 

Oddly, while there are certainly algorithms for computing the genus of the zero set of 
a given irreducible f'EQ[x,y] (e.g., [ Hoc94|l ), the effective geometric characterization 



of genus zero above appears to be new. So we present a proof of theorem [| in section 
3. 

In closing this first half of our introduction, we point out that our main theorems 
suggest that there is a deep connection between complex geometry and Diophantine 
complexity which has yet to be explored. In particular, we clearly need more refined 
geometric invariants to explicitly classify those curves (and surfaces) where we can 
hope to effectively study integral points. 

Main Theorems 2 and 3 are proved in section 4, and Main Theorem 1 is then proved 
in section 5. Some interesting open questions are briefly discussed in section 6. We 
now describe our necessary results on integral points more precisely. 

1.1. Curves with Many Integral Points. 

In this subsection, we will let / denote a polynomial in 1,[x,y}. Let us quote Siegel's 
classification of curves having infinitely many integral points. 



For convenience, we will sometimes use x and y in place of x\ and xi in the bivariate case. 
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Siegel's Theorem. [[Sie29| , |Lan83| , |Aya9 1| , |Sil9 || Let C be a curve defined over Z and 

irreducible over C. Then C has infinitely many integral points ==>■ C has genus zero 
and at most two distinct points in P|\C 2 . Furthermore, we have the following partial 
converse: Any C with genus zero and at most two distinct points at infinity will have 
infinitely many integral points in a sufficiently large finite algebraic extension of Z. 



Remark 2. By the genus of a variety V we will always mean the geometric genus of a 
smooth projective model for V . (Since geometric genus is a birational invariant, it will 
be independent of the chosen model.) The genus of a curve is described very nicely in 
Mir95|l , and the genera of higher dimensional varieties is defined in [|Har771 , |Kho78| . 



Remark 3. ForC = Sf, our definition ofC having a "point at infinity" is simply that 
the compactified zero set of C intersect P^\C 2 . So the "points at infinity" condition 
in Siegel's Theorem can actually be checked algorithmically, simply by considering the 
zero set of the homogeneous polynomial t deg f f(x/t,y/t)\ t =o in Qf\(0, 0). 

Note that an immediate corollary of Siegel's Theorem, and our preceding remarks, 
is that the condition on C in Main Theorem 1 can be replaced by the following: 

"...C has a component of genus zero with at most two distinct points at infinity." 

This version can be checked algorithmically, but gives a slightly larger exceptional 
case of 33V3 than the original condition. 

The final result on integral points we quote will allow us to efficiently decide a 
small, but highly non-trivial quantifier prefix. 

The JST Theorem. ||Jon81| , |Sch82| , Tun87|| The quantifier prefix V3 is decidable in 



polynomial time. More explicitly, given PEl,[x,y], we have that\/x 3y P(x,y) = iff 
all of the following conditions hold: 

1. The polynomial P factors into the form Po(x, y) Yli=i(y ~ Pi( x )) where Po(x, y) G 
Q[x, y] has no zeroes in the ring Q[x], and for all i, Pi G Q[x] and the leading 
coefficient of Pi is positive. 

2. Wx G {1, . . . , xo} 3y G N such that P(x, y) = 0, where xo = maxjsi, . . . , Sk}, and 
for all i, Si is the sum of the squares of the coefficients of Pi. 

3. Let d be the least positive integer such that dP\, . . . ,c?PfcGZ[x] and set Qi\ = dPi 
for all i. Then the union of the solutions of the following k congruences 

Qi(x) = mod d 



is all ofZ/dZ. 



Qk(x) = mod d 
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In particular, the above conditions can be checked in time polynomial in log(d), the 
heights of the coefficients, and the degree of P, via fast factorization of polynomials 
in <2 variables over Q and Z/dZ ||Coh93|| . ■ 



Remark 4. The JST Theorem can be strengthened slightly in the following way: one 
can replace d in condition (3) with any positive integer d' such that d'Pi, . . . , d'Pk G 
Z[x}. 

2. Geometric Background 

We first point out that a complete account of computability, decidability, and 
Turing machines can be found in [|GJ79| , Mat93| , [BCSS98|1 . Also, our notion of "input" 



will be fairly standard: either the sparse encoding of polynomials (over Z or Q) or 



the bit-wise encoding of algebraic numbers, for BSS machines over Z/2Z |[BCSS98 



Finally, for most of the basic facts we will use from algebraic geometry, we refer the 



reader to |Har77, Mum95, Bea96|. However, for the convenience of the reader, we 



will restate a few of the most central notions. 

Remark 5. Throughout most of this paper, "effectively computable" and "algorith- 
mic" will be taken to mean Turing computable. 

Returning to the concept of sparse encoding, the following notation will be useful: 
For any e G Z n , we let x e denote the monomial term x^x^ 2 ■ ■ -xfp. For any poly- 
nomial f(x) = J2e€i n c e(a)x e G C[ai, . . . , a m , Xi, . . . , x n ], we then let the support of 
/, Supp(/) C Z n , be the set of exponents {e G Z™ | c e (a) ^ 0}. (We are implicitly 
considering the Xi as variables to be solved for, and the aj as parameters we are free 
to choose.) Also, we will let the Newton polytope of /, Newt(/) C W 1 , be the 
convex hull of (i.e., the smallest convex set containing) Supp(/). 

An afRne variety is simply the complex zero set of a system of polynomial equa- 
tions. (So our varieties will not necessarily be reduced or irreducible | |Har77| , 
Mum95| |.) More generally, a quasi-affine variety is the set of complex points satisfy- 



ing any finite Boolean combination of polynomial equations and inequations. (Note 
that we mean ^, not < or <, when we say inequation.) In particular, when we 
say a set of indeterminates {a\, . . . ,a m } is chosen generically, we will mean that 
(ai, . . . ,a m ) G C m \W for some a priori fixed quasi-affine algebraic subvariety W 
(depending only on the property in question) with codimiy>l. For most purposes, 
assuming a property holds generically also implies that the property occurs with prob- 
ability 1. (A classification of a broad class of probability measures on C m for which 
this is true is not hard to derive.) We will also use the term variety collectively for 
affine and quasi-affine varieties. 

As for the geometric language of our main theorems, let us recall the following 
definitions: A morphism is simply a well-defined map from one variety to another, 
given by rational functions. When we relax the "well-defined" stipulation and allow 
our map to be undefined on a subvariety of codimension > 1, we then obtain a 
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rational map. Also, a birational map is a rational map with an inverse which is 
again a rational map. 

The inverse image of a point, for any given morphism, is usually called a fiber. 
For any curve C, we then say that a variety V is a bundle of curves fibered over 
C iff there is a morphism ip : V — > C such that every fiber of <p is a (not necessarily 
irreducible) curve. 



Definition 3. ||Bca96|| Assume temporarily that all varieties are irreducible, nonsin- 
gular, and compact. Let C be a curve. We then call a surface S ruled over C iff' 
there is a morphism ip : S — > C with every fiber isomorphic to P^. Similarly, we will 
call a surface S ruled iff there is a curve C for which S is ruled over C . Finally, an 
arbitrary algebraic surface S (not necessarily irreducible, nonsingular, or compact) is 



2 



said to be rational iff S is birationally equivalent to ¥ c . 

The following two related facts will prove useful. 

Theorem 5. [ [Kho78|| p| Suppose f G C[ai, . . . , a m , xi, . . . , x n ]. Then, for generic a, 
the genus ofSf a CC is exactly the number of lattice points in the interior o/Newt(/). 



Theorem 6. If S is a ruled surface, then its genus is zero. Also, if S is a bundle of 
curves (each with a genus zero component) fibered over another curve, then S has a 
component which is birationally equivalent to a ruled surface. ■ 



Theorem ^| follows easily from the development of [ |Bca96[| and , in particular, the 

,ces | Bca96 



classical Noether-Enriques theorem on algebraic surfaces [|Bea96 
Two interesting examples of theorem |5| are the following: 

Example 1. (3v3y C 3vVx3y) For any f 6 Z[i>, i/] \ {0} ; reconsider its zero set as 
a polynomial in Z[v,x,y]. Abusing notation slightly, let us denote this subvariety of 
C 3 by Sf. Then there is a natural projection <p from Sf onto the v-axis, and any 
fiber (p~ l (vo) is clearly of the form {v } x C where C is a finite union of lines. So 
the hypothesis of theorem [6| is satisfied in this example. Better still, the conclusion 
of theorem Q is easily verified: Sf is clearly birational to a ruled surface, since Sf is 
clearly a Cartesian product of a curve with a line. 

Example 2. (3u3v3y C 3u3vVx3y) For any f 6 Z[u, v , y] \ {0}, reconsider its zero 
set as a polynomial in Z[u,v,x,y]. Abusing notation once more, let us denote this 
subvariety o/C 4 by Sf. Then there is a natural projection ip from Sf onto the (u, v)- 
plane, and any fiber (y9 _1 (M , v ) is clearly of the form {(u ,v )} x C where C is a 
finite union of lines. More to the point, consider the inverse image of ip over a line L 
in the (it, v) -plane with positive rational slope. Clearly then, ip~ l (L) cSf is a surface 



The version of theorem || stated in [ Kho78| is actually a bit different, but easily implies our 



version here via an application of theorem |7j below. 
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S' fibered over L. In particular, every fiber v ? " 1 ( c ) H S' , for cEL, is clearly a curve 
with a genus zero component. So Sf contains a surface (S 1 ) of the type specified in 
Main Theorems 1 and 3, and by theorem this surface has a ruled component. 

To prove theorem ^ we will make some use of elimination theory, but in a geometric 
form. 

Theorem 7. Suppose V is a quasi-affine subvariety ofC m x C n defined over Q, and 
we respectively use coordinates a:= (a±, . . . , a m ) and ) for the first and 

second factors. Then the following assertions hold: 

1. ||Har77| , |Mum95| The set of a G C m for which there is an x G C n with (a, x) G 



V is another quasi-affine variety W C C m defined over Q. Furthermore, if 
we are given the sparse encodings of the polynomials defining V, then we can 
algorithmically determine the analogous data for W. 

2. ||Roj97| , |Roj98|| Given the sparse encodings of the polynomials defining a zero- 
dimensional variety U C Q , the sets U H Z n and [/ H N n can 5e effectively 
computed. 

3. | GM'1'89| . Chi96 | T/ie decomposition ofV into irreducible components, and their 



dimensions, can be effectively computed. 

A more general and explicit version of part (1) was derived by Tar ski in his work 
on quantifier elimination (over K) in the 1950's. Considerable improvements have 
since been made by other authors, giving singly exponential complexity bounds for 
the problem described in part (1). However, since our main concern is decidability, 
we will not dwell on these important extensions. 



Finally, we will need Hurwitz' Theorem [|Mir95| , (5il95| relating the genera of the 



domain and image of a morphism between curves. 

Theorem 8. Suppose (p : C — > C is a nonconstant morphism of nonsingular com- 
pact curves over C. Let g and g' respectively be the genera of C and C . Then the 
following relation holds: 

2g-2 = (de glf )(2g' - 2) + J>^(p) - 1) 

where the sum is over all points pEC such that (p is ramified at p, and e^(p) denotes 
the ramification index. ■ 

3. The Proof of Theorem f| 

We will first need the following definitions. 

Definition 4. Let MultiSub(Z n ) denote the set of all finite multisets of finite sub- 
sets of T, n . Let us also endow the following partial ordering on MultiSub(Z n ): De- 
clare . . . , S k } < {T h . . . , Ti} iff there are polynomials fx,... , f k , gi, . . . ,gi G 
C[xi, . . . ,x n ] such that k <l, Supp(/j) = Si for all i, Supp(gfj) = 1} for all j, and 
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Suppdli fi) — Supp(f| - gf). Concluding this connection to factoring, let us also de- 
fine the factor type, Tf GMultiSub(Z n ) ; of a polynomial /gC[xi, ■ ■ ■ ,x n ], to be the 
multiset of supports of its irreducible factors over C[x\, . . . , x n ]. 

It is not hard to see that for a given polynomial with parametric coefficients, pos- 
sessing a particular factor type determines a condition defining a quasi-afline variety. 

Lemma 1. Suppose f G Q[di, . . . ,a m ,Xi,... ,x n }. Then, for any Tg MultiSub(Z ?l ) , 
the set of all (ai, . . . , a m ) G C m for which f a has factor type T is a quasi-affine variety 
defined over Q. Furthermore, the polynomials defining this variety are effectively 
computable. 

Proof: First note that f a has factor type >{*Si, . . . , Sk} iff a set of equations involv- 
ing ai, . . . ,a m has a solution. This assertion is immediate, but for clarity we give the 
following example with k — 2: ai + a 2 x 2 + a 3 y 2 = (ati + j3ix + jxy) (a 2 + (3 2 x + 7 2 y) -<=>- 

the following system of equations has a solution (oti, (3i, 71, a 2 , fa, 72) GQ : 

(X\OL2 — a<i = 
P1P2 ~a 2 = 
7i72 -a 3 = 
ai(3 2 + a 2 [3i = 
etc... 

So by theorem [7|, possessing a factor type above or equal to {Si, . . . , Sk} defines 
a quasi-affine subvariety of values of a. Now note that the poset of possible factor 
types for any fixed / is finite, and recall that quasi-affine varieties are closed under 
any finite sequence of Boolean operations. So by another application of theorem [7|, 
the set of a for which /„ has factor type exactly {Si, . . . , Sk} is also a quasi-affine 
variety. So we are done. ■ 

We can now at last prove theorem f|. 
Proof of Theorem ^} Recall once again that quasi-affine varieties (even those 
defined over Q) are closed under any finite sequence of Boolean operations. So it 
suffices to prove that some collection of algebraic functions of the a in question 
form a (Turing computable) quasi-affine variety. So by lemma 1, it thus suffices to 
assume that g is an irreducible polynomial in C[ai, . . . , a m , x, y], and a G Q is such 
that g a is irreducible. 

Let C be the complex zero set of g a . Then C is an irreducible curve, possibly 
with singularities. The singularities of C are precisely the zero set (in the (x, y)- 

plane Q ) of an effectively constructible system of polynomials in Q[ai, . . . ,a m ,x,y] 
|[Har77| , [Mum95|| . So the coordinates of every singular point are algebraic functions 
of ai, . . . ,a m defined over Q. 

Since the number of such singularities is finite, a finite sequence of blow-ups will 
give us a new curve C birationally equivalent to C. Furthermore, by our preceding 
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observations, and the structure of the blow-up map [|Har77| , Mum95 |, the coefficients 



of C are algebraic functions of defined over Q, as well 

To conclude, note that C is a curve in P^ , where N is 2 plus the sum of the orders 

of the singularities of C. Furthermore, we can consider the ambient projective plane 

in which C lies as a coordinate subspace of P^ . So let <p be the natural projection 

mapping PjjY onto the x-axis of this copy of P; 

Let us apply theorem [| to the preceding morphism if : C — > P^. We then obtain 

that g a has genus zero iff 



2deg^ = 2 + ^(e„(p)-l). 

Now by theorem |7| and our preceding observations (as well as the classical notion of 
discriminant), we can write the preceding sum of ramification indices as the order of 
vanishing of some (effectively constructible) polynomial in Q[a%, . . . ,a m ] at a point. 
Furthermore, this order of vanishing is equal to some fixed constant iff a lies in a 
quasi- affine variety (defined over Q) depending on the constant. Similarly, the degree 
of if is some fixed constant iff a lies in a quasi- affine variety depending on the constant. 
We thus at last obtain that Q g is a Turing constructible quasi-affine variety. ■ 

4. Deciding Prefixes Ending in V3 

To prove Main Theorems 2 and 3, we will first describe a construction which is 
common to both proofs. So let us temporarily consider polynomials in Z[tt, v, x, y]. 
More precisely, it will be helpful to consider / as a polynomial in x and y with coef- 
ficients in Z[u, v]. For emphasis, we will now respectively write f( u ,v) an d /(«,«) (x, y) 
in place of / and f(u,v,x,y). 

Definition 5. For any f 6 Z[u, v, x, y], lefEf be the set of all pairs (w,u)eN 2 such 
that 

Vx 3y with f( u , v )(x,y) = Q. 

Our main trick for proving Main Theorems 2 and 3 is the following: create an 
explicit quasi-affine variety Qf C C 2 defined over Q, whose positive integral points 
contain (and very nearly equal) Hf. The following definition and lemma will clarify 
our complex geometric approximation. 

Definition 6. Following the notation of definition ^ for any f e C[u,v,x,y] let 
Qf'-=Gf, where we consider f as a polynomial in x and y with coefficients in C[u,v]. 
So Qf C C 2 and the polynomials defining Qf lie in C[u,v). 

Lemma 2. The set H/ is contained in Qf D N 2 . In particular, dimQf > 1 iff f lies 
in the exceptional locus defined in Main Theorem 3. 

Furthermore, if we restrict to f EC[v,x,y] (and thus consider S / CN and O/CCj, 
we have that S/ C Qf fl N. In particular, under this restriction, dim £7/ > 1 iff f lies 
in the exceptional locus defined in Main Theorem 2. 
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Proof of Lemma [2]: By Siegel's Theorem, (u, v) G ==>- Sf {uv) contains an irre- 
ducible curve of genus zero. So the inclusion 5/ CO/ D N 2 is clear. The condition for 
dimO/ > 1 is then just a reformulation of the condition that enough specializations 
of (u, v) make Sf, v . have genus zero. The refined statements for when / G Z[t>, x, y] 
follow similarly. ■ 

The description of as a subset of the positive integral points on a quasi-afline 
variety allows an intuitive complex geometric approach to constructing algorithms 
for a large family of special cases of Diophantine prefixes such as 3V3 and 33V3. In 
particular, theorem [| tells us that our quasi-affine variety Qf is effectively computable, 
and a judicious use of computational algebra can make this implementable. 

Remark 6. We now briefly clarify the statement of "genericity" in Main Theorems 
2 and 3: Fix the Newton polytope PcR 3 of f. Then, by theorems [| and and 
the classical Bertini's theorem ||Har77| , |Mum95|| , Sf will be an irreducible non-ruled 



surface, provided the coefficients are chosen generically and P has at least one lattice 
point in its interior. Thus ( except for a meager family of supports ) Main Theorem 2 
implies that for any fixed support we can decide 3V3 for a Zariski-dense set of f . The 
analogous statement for Main Theorem 3 can be derived in exactly the same way. 

Proof of Main Theorem 2: To construct our necessary algorithm, note that as 
observed in earlier situations, the variable u no longer occurs in /. So, since the 
prefixes 3u3vix3y and 3vix3y are identical for such /, we may now consider Qf as 
a subvariety of C. (And the polynomials defining Qf lie in Q[f].) Let us also assume 
/ is not identically zero. (For when / is identically zero, the prefix in question is 
trivially true.) 

Clearly then, if dimO/ < 0, deciding whether 3v Wx 3y such that f(v,x,y) = 
reduces to simply checking a finite number of instances of the prefix V3. (Recall also 
that we can effectively detect dimfiy <0 by theorems [| and By the JST Theorem 
and theorem |7|, we thus need only show that the hypothesis of Main Theorem 2 implies 
that dimf2/<0. But this follows immediately from lemma 2. ■ 

The proof of Main Theorem 3 is almost exactly the same, save for the fact that the 
polynomials defining Qf lie in Q[u, v\. So we will omit the proof of Main Theorem 3 
and go directly to the proof of our first main theorem. 



5. The Proof of Main Theorem 1 

Let us temporarily assume that Big N 2 is computable. Let us also temporarily 
assume that the stated special case of 33V3 is decidable. To derive a contradiction, 
we will construct an explicit algorithm to decide the prefix 33V3 in general. To do 
this, we will again use our algebraic geometric trick from the last section. 

Accordingly, our algorithm will have three cases, dictated by the topology of Qf in 
C 2 . Also note that 3u3vVx3y is trivially true when / is identically zero, so we may 
assume that / is not indentically zero. 
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By theorem |], the JST Theorem, and theorem ^ once again, we know that the 
prefix V3 is sufficiently well-behaved so that we can make a simplification: we may 
assume that flf is irreducible. Also, using part (3) of theorem [7|, it is clear that Cases 
I, II, and III below can be distinguished effectively. So let us now solve these cases 
individually. 
Case I: ft f = 

By our preceding observations, we immediately obtain that 

3u 3t> Vx 3y f(u, v, x,y)=0 

is false. ■ 

Case II: dimf^ = 

Here, we need only check one instance of V3. By the JST Theorem, we can do this 
in polynomial time, so we are done. ■ 
Case III: dim Q f > 1 

By assumption we can solve the case where dimf2j = 2. (Indeed, when dimfij = 2, we 
can certainly find a curve in Qf satisfying the properties required in statement (2) of 
Main Theorem 1.) So let us assume dimf2/ = l and, to simplify notation slightly, let 
C :=Qf. We are left with just two subcases to consider and, by assumption, we can 
compute Big N 2 to effectively distinguish them. 

Case 111(a): C has finitely many positive integral points 
Since we can compute Big N 2 , we can simply enumerate all possible positive integral 
points and use the JST Theorem a finite (but most likely huge) number of times to 
decide 33V3. ■ 

Case 111(b): C has infinitely many positive integral points 

By our initial assumption, this case of 33V3 is tractable as well. ■ 
Having thus obtained an algorithm contradicting the MR Theorem, we are done. ■ 



6. Conclusion 



We have seen a geometric construction which implies a weak version of the state- 
ment "7iT'P N (3) is decidable =>- Big N2 is uncomputable." The decidability of 
Hilbert's Tenth Problem in three variables is still open, as is the existence of com- 
putable general upper bounds on the size of integral points on algebraic curves. So 
knowing the decidability of HTVfq(3) or the computability of Big N2 would have pro- 
found implications in algorithmic number theory, not to mention arithmetic geometry. 

We emphasize, however, that the uncomputability of Big z 2 would by no means 
contradict the effective upper bounds (for heights of integral points) which have al- 
ready been found [Bak68| , |Bak69| , PC70| , |Bri84| , Pou93|| for certain special classes of 
curves. More precisely, should Big z 2 eventually prove uncomputable, we obtain from 
our development that at least one of the following statements must be true: 
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(A) Effective upper bounds on integral points must cease to exist for some infinite 
class of non-superelliptic curves of genus at least two.[] 

(B) Detecting infinitudes of integral points on a curve of genus zero is undecidable. 

Furthermore, assuming the decidability of detecting infinitudes of rational points on 
curves of genus < 1 (and the falsity of statement (B)), the uncomputability of Big z 2 
would also immediately imply the uncomputability of Big Q 2 . 

It is also clearly the case that the uncomputability of Big z 2 would not contradict 
the decidability of HTVz(2), should the latter statement prove true. Indeed, the 
uncomputability of Big z 2 would only rule out a stronger version of the decidability 
of H,TVz{2) — the determination of all integral points when there are only finitely 
many. More to the point, the existence of effective general upper bounds on the 
height of the smallest integral point is still an open question. For example, Steve 
Smale has conjectured that such upper bounds, for curves of positive genus, exist and 



will be singly exponential in the size of the dense encoding ||Sma98|| . So the truth of 
Smale's conjecture would immediately imply a brute force algorithm for the positive 
genus case of 7YT"Pz(2). 

We also point out that the exceptional locus in Main Theorem 3 can be pared down 
somewhat: Via a suggestion of Smale, one can sometimes assume additionally that 
the forbidden / have a zero set which is either (a) reducible or (b) irreducible and 
singular. This refinement is based on examining the critical values of (the restriction 
to Sf of) the natural projection mapping C 4 to the (u, v )-plane. Other refinements 
based on a closer examination of the real part of Sf are possible and will be mentioned 
in future work. 

We will close by stating a few conjectures and open problems related to our de- 
velopment. First note that if the statement "Big N2 is computable •<=>- Big Z2 is 
computable" were true, then we could strengthen Main Theorem 1. (In particular, 
we could replace Big N2 by Big Z 2 .) Toward this end, we make the following conjec- 
ture on a type of equidistribution for integral points on the real part of a genus zero 
curve. 

Conjecture 1. Suppose CcC 2 is a curve defined over Z and irreducible over C. 
Suppose further that some irreducible component Cr of C PlR 2 has noncompact inter- 
section with the first quadrant. Then Cr has infinitely many integral points ==>- Cr 
has infinitely many positive integral points. 

The truth of this conjecture, combined with a little quantifier elimination over R, 
would immediately imply the aforementioned equivalence of Big N2 and Big Z2 . 

However, a potentially harder problem is to refine our proof of Main Theorem 1 to 
yield the truth of the following conjecture we have been alluding to. 

Conjecture 2. H.TVn(3) is decidable ==> Big N2 is uncomputable. 



6 Recalling that every curve over C of genus two is hyperelliptic [Mir95|, the results of [Gra94, 
Poo96| give evidence that this lower bound might need to be increased to three. 
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In particular, a refinement of our geometric approach seems possible, but quite 
subtle. For example, the proof of a 1970 theorem which essentially computes Big z 2 



in the special case of genus one curves [BC70|| involves constructing a very special 



birational map. The map Baker and Coates construct takes an arbitrary genus one 
curve to a curve in Weierstrass normal form, preserves rational points, and almost 
preserves integral points. The structure of their map was sufficiently good so that 
they could use the previously known bounds for curves in Weierstrass normal form 
to derive height bounds for the original (possibly more general) genus one curve. 

An analogous construction could be attempted for reducing the exceptional locus 
of Main Theorems 1 and 3 to the prefix 333. For instance, one could try to use a 
birational map sending Sf to a ruled surface with certain special properties. Such a 
construction, if done properly, could be used to prove the undecidability of 333 or 
the equivalence of the decidabilities of 3V3 and 33. Unfortunately, as of 1998, not 
enough is known about integral points on ruled surfaces, or even rational surfaces, to 
make this approach easy. Nevertheless, we hope to address this point in the future. 

We also propose the following conjecture motivated by our results. 

Conjecture 3. The prefixes 3V3 and 33 are decidable. However, Big N 2 is uncom- 
putable and TCTV^(3) is undecidable. 

The author is also willing to offer $1000 (US) for the first correct published proof of 
the decidability of HTV?q(3). This will hopefully prove a safe wager. 

Finally, we remark that for the sake of simplicity, we have not given the best 
possible complexity bounds. It is therefore quite likely that Main Theorems 2 and 3 
can be improved to give algorithms which run in doubly exponential time. In fact, 
we are willing to conjecture more. 

Conjecture 4. The Diophantine prefixes 3V3 and 33V3 are both generically decid- 
able within singly exponential time. 
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NOTE ADDED IN PROOF 



Joseph H. Silverman has just proved [ |Sil9J| my Conjecture 1 above, so we now 
have the equivalence of the computabilities of Big N 2 and Big Z2 ! We can thus now 
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strengthen Main Theorem 1 (and sharpen Conjectures 2 and 3) by replacing Big 
with Big z 2 throughout. 
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